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ABSTRACT 



A feedback-based adaptive network is described wherein at 
least a portion of the network elements report operating 
information relating to network conditions to a centralized 
data store. The information which is reported to the data 
store is analyzed by a policy engine which includes a 
plurality of application specific plug-in policies for analyz- 
ing selected information from the data store and for com- 
puting updated control information based upon the analysis 
of the information. The updated control information is fed 
back to selected network elements to thereby affect opera- 
tion of the selected elements. Typically, when the operation 
of a network element has been affected, its corresponding 
operating information will change. The new or changed 
network element operating information is then reported to 
the data store and analyzed by the policy engine. The policy 
engine may then generate new or updated control informa- 
tion for affecting the operation of selected elements in the 
network. In this way, the dynamic and automatic feedback 
control of network elements is provided in order to allow the 
network to adapt to changing conditions. Events relating to 
changing conditions in the network may be reported to 
selected elements in the network using an event notification 
service. Additionally the adaptive, feedback-based network 
of the present invention may include a network quality 
monitoring system for evaluating performance characteris- 
tics or other aspects of the network based upon predeter- 
mined standards or criteria. If it is determined that a par- 
ticular characteristic of the network does not conform with 
the standards established for that characteristic, the policy 
which controls that particular characteristic of the network 
may be automatically and dynamically modified to thereby 
affect the network performance. 

20 Claims, 20 Drawing Sheets 
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TECHNIQUE FOR PROVIDING AUTOMATIC The use of networks such as LANs or WANs allows data 

EVENT NOTIFICATION OF CHANGING or other information to be exchanged over large distances at 

NETWORK CONDITIONS TO NETWORK a reasonable speed, and at relatively low cost. Although 

ELEMENTS IN AN ADAPTIVE, FEEDBACK- conventional network systems are advantageous for a vari- 

BASED DATA NETWORK S ety of reasons, implementing and maintaining such networks 

presents an ongoing problem. Moreover, a network may 

RELATED APPLICATION DATA span a plurality of separate LANs and/or WANs to create the 

This invention is related to U.S. patent application Ser. impression of a unified network. This has greatly compli- 

No. 09/342,743, filed on the same day as this patent cated the implementation, maintenance, and control of sys- 

application, naming Shankar Nataraj an, Andrew G. Harvey, 10 tem parameters within the network. 

Hsuan-Chung Lee, Vipin Rawat, and Leo Pereira as As the use of computer networks proliferates, there exists 

inventors, and entitled "Dynamically Adaptive Network an increasing need to improve computer network designs 

Element in a Feedback-Based Data Network". That appli- and implementations in order to facilitate the management, 

cation is incorporated herein by reference in its entirety and implementation, and modification of such networks, 

for all purposes. is 

This invention is related to U.S. patent application Ser. SUMMARY OF THE INVENTION 

No 09/342,337, filed on the same day as this patent According to spec ific embodiments of the invention, a 

application, naming Shankar Natarajan, Andrew G. Harvey, tcchniquc ^ providcd for implementing a feedback-based 

Hsuan-Chung Lee, Vipin Rawat, and Leo Pereira as da t a network which is able to automatically and dynamically 

inventors, and entitled 'Policy Engine which Supports ™ momtorcharac teristics of various aspects of the network and 

Application Specific Plug-ins for Enforcing Policies in a ad t to chan ^ network conditions by dynamically and 

Feedback-Based, Adaptive Data Network'. That application automatically modifying selected network parameters in 

is incorporated herein by reference in its entirety and for all order tQ achieye a desked performaDce leveL 

purposes. . . „ , , T „ , . . „ 25 According to specific embodiments of the invention, a 

k T ^7r,T l ^ re , latCd ^ U " S * Pat ^ Dt apphc u atl0n Ser method and computer program product are disclosed for 

No 09/S42,321, filed on the same day as this patent idin cvcnt notification of dynamic feedback control 

application, naming Shankar Natarajan, Andrew G. Harvey, £ata to network elements in a data network. Each of the 

Hsuan-Chung Lee Vipin Rawat, and Leo Pereira as nctwork elemcnts faas a ^ of ^ parameters 

inventors, and entitled A System For Dynarmcally And ^ ociait6 therewith . The operating parameters associated 

Automatically Updating Control Parameters Of Network ^ a nc(work elemcQt afc rcktcd tQ a( ^ Qnc 

Elements After Analyzing Network Performance Based eter of lhat element> At least one data mm 

Upon Selected Guidelines' . That application is incorporated fc monitorcd for desircd mformation rela(ing to Detwork 

herein by reference in its entirety and for all purposes. conditions. U an event is detected relating to desired network 

This invention is related to U.S. patent application Ser. 35 condition information, at least one selected network element 
No. 09/342,742, filed on the same day as this patent is notified of the event. An additional aspect of this embodi- 
application, naming Shankar Natarajan, Andrew G. Harvey, ment p rov i de s that the desired information relates to updated 
Hsuan-Chung Lee, Vipin Rawat, and Leo Pereira as control information generated in response to changing con- 
inventors, and entitled "Technique for Collecting Operating mQn& m the net work. When the availability of updated 
information from Network Elements, and for Controlling 4Q control information is detected, at least one network element 
Network Element Behavior in a Feedback-Based, Adaptive may be notified 0 f this event. The network element may then 
Data Network". That application is incorporated herein by respond to the event notification information by automati- 
reference in its entirety and for all purposes. cally configuring itself using the updated control informa- 

This invention is related to U.S. patent application Ser. ti 0 n to thereby adapt to the changing conditions in the 

No. 09/342,341, filed on the same day as this patent 45 network. 

application, naming Shankar Natarajan, Andrew G. Harvey, Another embodiment of the present invention is directed 

Hsuan-Chung Lee, Vipin Rawat, and Leo Pereira as to m cvcnt notification system for providing event notifi- 

inventors, and entitled "Technique for Providing Dynamic cation of dynamic feedback control data to network ele- 

Modification of Application Specific Policies in a Feedback- mcms in a ^ nciW0T ^ E acn 0 f tne network elements has 

Based, Adaptive Data Network". That application is incor- 50 a plurality of operating parameters associated therewith. The 

porated herein by reference in its entirety and for all pur- operating parameters associated with a particular network 

P oscs - element are related to at least one control parameter of that 

BACKGROUND OF THE INVENTION element. The event notification system comprises at least 

one event supplier entity for monitoring at least one data 

1. Field of the Invention 55 source for desired information relating to updated network 
The present invention relates generally to data networks, conditions. The event notification system further comprises 

and more specifically to a technique for providing a a i i eas t one event handler entity for notifying at least one 

feedback-based data network for dynamically adapting to network element of an event relating to updated network 

changing network conditions. conditions. The event notification system further comprises 

2. Background 60 an event server operable to receive event notification infor- 
The merging of computers and communications has had mation from the event supplier entity. The event server is 

a profound influence on the way computer systems are further operable to communicate with the event handler 

organized. As computer technology advances, the role of entity to cause the event handler entity to alert the at least 

computer networks steadily increases. Various types of one network element of an event relating to updated network 

computer networks include internetworks (e.g., the 65 conditions. In one embodiment of the present invention, the 

Internet), local area networks (LANs), metropolitan area event handler entity may be configured to also perform the 

networks (MANs), and wide area networks (WANs). function of the event supplier entity. An additional aspect of 
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this embodiment provides that, where the desired network FIG. 5C shows a schematic block diagram of policy 

condition information relates to updated control information engine 254 of FIG. 2, in accordance with a specific embodi- 

generated in response to changing conditions in the network, menl of the present invention. 

the event handler is further operable to cause the associated FIG. 6 shows an example of a flow diagram of an 

network element to retrieve and store at least a portion of the s initialization procedure for initialising the network of FIG. 

updated control information. The network element may then 2. 

automatically configure itself utilizing the updated control nG 7 shows a flow diagram of m event registration 

information to thereby adapt to the changing network con- procedure 700 in accordance with a specific embodiment of 

ditions * the present invention. 

Further embodiments of the present invention are directed 10 nQ g a flQW of a netW ork element 

to a method and computer program product for providing reporting procedure 800 in accordance with a specific 

event notification of network conditions to network devices embodiment of the present invention, 

in a feedback-based adaptive data network. At least one first FIG. 9A shows a flow diagram of an event notification 

network device is monitored for mformation relating to a dmc 900Am accordance ^ a specific embodiment 

condition and/or status of that first device. The condition 15 ^ me Dresent invention 

and/or status of the first device is then reported or notified to nn , n ' ,. , . n 

at least one other network device. An additional aspect of nG - 9B ^ 3 1™^™ °[ an even * DOtlfica ^ n 

this embodiment provides that, where the condition or status ^P™ 56 P™edure 900B m accordance with a specific 

of the first device relates to an error detected by the first embodiment of the present invention. 

device, an event notification message relating to the error is 20 FIG. 10 shows a flow diagram of a network element 

reported to at least one other network device. parameter update procedure 1000 in accordance with a 

Another embodiment of the present invention is directed s P ecific embodiment of the present invention, 

to an event notification system for providing event notifi- ^G. 11 shows a specific embodiment of a flow diagram 

cation of network conditions to network elements in a data of a P olic y engine analysis procedure 1100, which may be 

network. The event notification system comprises a plurality 25 implemented by the policy engine 254 of FIG. 2. 

of event handler entities, wherein each event handler entity FIG. 12 shows a flow diagram example of a frame relay 

is associated with a respective network element. The event CIR policy 1200 which may be implemented by procedure 

notification system further comprises an event notification 1100 of FIG. 11. 

server operable to receive information from each of the FIG. 13 shows a specific embodiment of a flow diagram 

event handler entities relating to a condition of its respective 30 for implementing a network performance quality monitor 

network element. The event server is further operable to procedure 1300, which may be implemented by monitor 

report event information relating to the condition of at least system 262 of FIG. 2. 

one of the network elements to selected event handler fig. 14 shows a specific embodiment of a flow diagram 

entities. Each of the event handler entities is operable to f or implementing an administration procedure 1400, which 

receive event notification messages from the event notifica- 35 m ay be implemented by ADMIN system 260 of FIG. 2. 

tion server. Further, each of the event handler entities is pj G 15 shows m example of a flow diagram for imple . 

operable to report information relating to a condition of its meming a data store event handler reporling proce dure 

respective network element to the event notification server. 1500> which may be implemented by data store 252 of FIG. 

In accordance with this embodiment, information relating to 2 

errors or other network conditions which are detected by a 40 1* 1* j \o •« . * u *■ j* — a 

- 4 . , . A , A ., A , A , . t J . FIGS. 16, 17 and 18 illustrate schematic diagrams and 

first network element may be distributed to other network a . r i t. lL 

. , . lL * n . . flow diagrams, showing, by way or example, how the 

elements via the event notification system. c „ , T 1*^1 1 r .u . • 

J feedback-based adaptive network of the present invention 
Additional features and advantages of the present inven- may be imp i em ented in a network for automatically adap- 
tion will become apparent from the following descriptions of mg {Q cnanges m Qetwork conditions . 
its preferred embodiments, which descriptions should be 

taken in conjunction with the accompanying drawings. DETAILED DESCRIPTION OF THE 

^r^™™^ „r ™^ ™ *„, T . T ™ PREFERRED EMBODIMENTS 
BRIEF DESCRIPTION OF THE DRAWINGS 

. , .ru.-i.ii_i- c FIG. 1 shows an example of a frame relay network. The 

HG.lshowsanexanipleofaschematicbloc^ 5o frame ^ netWQrk be m h{ q{ flS ^ a 

a conventional frame relay network 150. grQup of frame reky virtual circuits fo . data 

FIG. 2 shows a schematic block diagram of a specific between a first user/computer system and a second user/ 

embodiment of the feedback-based, adaptive data network computer system. A frame relay virtual circuit represents a 

of the present invention. high-speed packet switching path used in wide area net- 

FIG. 3 shows a schematic block diagram of the adaptive 5S it has become very popular for I_AN to LAN 

feedback-based data network of FIG. 2, implemented as a connections across remote distances. A consumer may lease 

frame relay network. a frame relay virtual circuit between two points and send 

FIG. 4 shows a schematic block diagram of an alternate packets of information between the two points. It is also 

embodiment of the feedback-based, adaptive data network possible to lease frame relay circuits between a given site 

of the present invention. 60 and multiple other sites. 

FIG. 5A shows a schematic block diagram of a router 10 As shown in FIG. 1, a frame relay network with one frame 

which may be implemented as a network element in accor- relay virtual circuit 150 connects a client computer 102 with 

dance with the feedback-based, adaptive data network of the a server 114. Typically a service provider charges for access 

present invention. at both ends of the network. When leasing the frame relay 

FIG. 5B shows a schematic block diagram of data store 65 virtual circuit, the consumer agrees to pay a specified 
252 of FIG. 2, in accordance with a specific embodiment of amount of money in return for the service provider guaran- 
tiee present invention. teeing a time-averaged bandwidth for the virtual circuit. 
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One difference between a physical leased line and a frame vide virtual circuits, each circuit having a specified CIR and 
relay circuit is that, with the physical leased line, a user pays EIR value. When a service provider provisions a virtual 
for a specified amount of bandwidth, and may send traffic all circuit having specific CIR and EIR values, a human tech- 
day long at the maximum speed. However, with a virtual nician is hired to provision the circuit by statically config- 
circuit, data bursts may be sent at full speed, but the 5 uring specific network elements within the cloud to form the 
long-term average usage (i.e., committed information rate or desired communication path having the specified CIR and 
CIR) must be at or above a predetermined level. EIR values. 

Typically, when a client is provisioned a virtual circuit, a Conventionally, the provisioning of a virtual circuit by a 

service provider will guarantee to provide a minimum service provider requires not only manual configuration of 

amount of bandwidth for handling client traffic. More 10 frame relay devices 104A, 104B, but also requires coordi- 

specifically, the consumer pays for, and the service provider nation between the end user and the service provider which 

agrees to provide, a virtual circuit which is able to handle a manages the FR cloud 110. This coordination usually takes 

specified committed burst size (Be) of data (e.g. Bc=256K P lace °y sending a request to the service provider to imple- 

bits) over a specified committed time interval (Tc) (e.g. Tc=l ment or modif y a circuil within PR cloud 110 0nce 

second). The value Bc/Tc is referred to as the committed 15 the request is received by the service provider it is handled 

information rate (CIR), which specifies the rate at which the manual . 1 y ^ a . se 1 rviC€ technician who statically configures 

virtual circuit will accept data from the consumer. In the the various switches and other devices within FR cloud 

1 u * L r^in 1 1 * u * / HO to implement a virtual circuit having specified CIR and 

example above, the CIR value is equal to 256K bits/sec, £ maQual * £ 

meaning that over the span of one second, the service uncommonfor days to pass before the consumer (e.g., client 

provider guarantees to handle 256 Kbits of data. M m) fa able {Q ^ m / virtual dia ^ l 

At times, however, a client may be running an application Moreover, significant delays may be experienced when 

which requires more than 256K bits/sec of bandwidth. For attempting to modify or reconfigure parameters associated 

example, the client may be required to send bursts of traffic a virtual circuit. For example, let us suppose that the 

at 300K bits/sec from time to time. Although the CIR value consumer initiates a video conferencing application between 

represents the average guaranteed bandwidth for the virtual 25 client 102 and server 114. Further, let us suppose that the 

circuit, the service provider also provides an allowance for current CIR and EIR bandwidth values for virtual circuit 150 

data bursts. This allowance is referred to as the excess burst are insufficient for supporting high quality video/voice 

size (Be) (e.g. Be=64 Kbits). Thus, using the example above, applications, resulting in the client 102 receiving poor 

in time Tc (1 second), the Be value (256K) may be exceeded quality video images and voice information from server 114. 

by Be (64K) without the customer having to pay. This 30 In order to overcome this problem, a human operator at the 

amount of additional bandwidth is referred to as the excess client side may send a request to the service provider to 

information rate (EIR=Be/Tc), and represents the excess increase the CIR value for the virtual circuit 150. 

bandwidth which the service provider will attempt to deliver However, in order to increase the CIR and/or EIR values 

to the consumer. Any data sent by the consumer in excess of on the virtual circuit 150, a plurality of network elements 

the EIR value will be dropped. Further, data bursts which are 35 within the WAN which form the communication path 

between the Be and Be value may be discarded in the event between client machine 102 and server 114 must be stati- 

of network congestion. cally reconfigured by a human operator to support the new 

The frame relay virtual circuit 150 of FIG. 1 comprises a CIR and EIR values. Thus, when the request from the human 
plurality of individual network elements (e.g., 104A, 104B, operator at client side is received at the service provider end, 
108) which are part of a wide area network. Typically, 40 a system administrator or other service technician must 
different portions of the wide area network are owned and/or statically reconfigure each of the network elements which 
managed by separate entities. For example, the service form the virtual circuit connecting client 102 with server 
provider may manage or control the gateways (104A, 104B) 114. This may include statically reconfiguring frame relay 
for accessing the wide area network from either the client devices 104A and 104B (via local console/terminals 106A, 
machine 102 or server machine 114. More typically, 45 106B, respectively), and also may include reconfiguring the 
however, the gateways are controlled by the end user who appropriate switches 108 and/or other devices which are part 
has the ability to reconfigure these devices. Gateway devices of FR cloud. This manually implemented, time consuming 
104A and 104B may, in turn, communicate with frame relay procedure is extremely undesirable, 
cloud 110, which may be managed or controlled by a Additionally, it should be noted that parameters in con- 
separate entity such as, for example, a data network com- 50 ventional frame relay network devices are typically derived 
pany (e.g., AT&T, MCI, Sprint, etc.). Typically, the frame from a static on-board configuration having minimal inter- 
relay service provider and the frame relay carrier are the action with the provisioning switching device in the network 
same entity. and no interaction with the far end device that terminates the 

In the example of FIG. 1, devices 104A and 104B circuit. The presence of various network elements in the 

represent frame relay devices, such as, for example, routers 55 network implies that no network element is an isolated 

which are configured to handle frame relay protocol. Each of entity, but rather is affected by other network elements that 

the frame relay devices 104A, 104B are typically statically surround it. Accordingly, control actions performed on one 

configured by a system administrator using a command line network element can affect other network elements. Since 

interface (CLI) via a local console (106 A, 10 6B, the performance of the "switching fabric" can affect the 

respectively). Each terminal 106A, 106B is connected 60 performance of the "edge devices", it is preferable to pro- 

directly to its respective Frame relay device via a local vide a network model which is able to accommodate the 

console port (not shown) at the device. The virtual circuit "multi-variable w nature of networks and implement a control 

between frame relay device 104A and frame relay device scheme wherein information from at least a portion of 

104B is provided by the FR cloud network 110, maintained related network elements is collected before a management 

and controlled by the frame relay service provider. The FR 65 or control decision is made at a particular network element, 

cloud includes a plurality of switching devices (108) which Using the technique of the present invention, a dynamic 

are statically configured by a system administrator to pro- feedback-based adaptive network may be provided for auto- 
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matically detecting a client's need for increased bandwidth, a conventional network. For example, in the embodiment of 
and for automatically and dynamically reconfiguring the FIG. 2, the control elements include data store 252, monitor 
appropriate network elements to provide sufficient band- system 262, ADMIN system 260, policy engine 254, and 
width on the virtual circuit to support the user's current event server 270. Additionally, as used in this application, 
applications). The feedback-based adaptive network of the 5 the term "network element" refers to any hardware or 
present invention monitors current conditions of local and/or software component of the adaptive feedback-based net- 
remote network elements and dynamically adjusts network work of the present invention, including control elements, 
control parameters based upon analysis of the monitored The network element may be a logical or physical entity 
network elements. A specific embodiment of the dynami- within the network which is being managed or controlled, 
cally configurable feedback-based adaptive network of the The network element is preferably designed with built-in 
present invention is shown in FIG. 2 of the drawings. instrumentation which allows for the collection of relevant 
The feedback-based adaptive network of the present information which may be subsequently used to determine 
invention utilizes a technique wherein at least a portion of control actions to be applied to the network clement. In the 
the network elements (e.g., 204A, 204B, 208A, 208B, etc.) network of FIG. 2, network element 204A may be any 
report network information relating to network conditions to 15 hardware or software component which has a measurable 
a centralized data storage entity (e.g., data store 252). The parameter that can be reported. Examples of network ele- 
reported data corresponds to information relating to the mcnts include routers, switches, hosts, modems, terminals, 
current condition or status of each of the reporting network dial access servers, gateways, ports, channels, interfaces, 
elements in the network. The information which is reported circuits, processes, drivers, protocols, services, applications, 
to the data store 252 is analyzed by a policy engine 254. The ^ etc. 

policy engine 254 includes a plurality of application specific In accordance with a specific embodiment, a network 

plug-in policies for analyzing application specific informa- clement which has been configured to support the feedback- 

tion from the data store and for computing updated control based adaptive configuration technique of the present inven- 

information based upon the analysis of the information. The tion will include a monitor/agent (not shown) for 

updated control information may include any type of ^$ monitoring, measuring and/or computing information (e.g. 

information, parameters, and/or actions which may be used state data) and for reporting this information to the data store 

to affect the operation of one or more network elements. The 252. The monitor/agent may be implemented via hardware, 

updated control information is then fed back to selected software, or a hybrid combination of hardware and software, 

network elements to thereby affect operation of the selected The reported information may be used by the policy engine 

elements and/or network. Typically, when the operation of a 30 254 to calculate updated control information for effecting 

network element has been affected, its corresponding oper- the operation of the network element 204A. As described in 

ating parameters and/or operating information will change. greater detail below, there are a number of ways in which the 

The changed operating parameters are then reported to the updated control information may be fed back into the 

data store 252 and analyzed by the policy engine 254. The network element. Once received, the network element will 

policy engine may then generate new or updated control 35 automatically configure itself utilizing the updated control 

information or parameters for affecting the operation of information to thereby affect its operation, which may 

selected elements in the network. In this way, the network of include affecting one or more operational parameters of the 

FIG. 2 is configured to adapt to changing conditions in the network elements. 

network by providing a dynamic feedback mechanism. i n a specific embodiment where the network element is 

Using this dynamic feedback mechanism, selected network 40 part 0 f a frame relay circuit, examples of the information 

elements may be dynamically and automatically reconfig- reported by the network element may include information 

ured to cause the performance of various aspects of the relating to: committed information rate (CIR), excess infor- 

network to conform with desired performance criteria. mation rate (EIR), committed burst size (Be), excess burst 

The feedback-based adaptive network of FIG. 2 includes size (Be), congestion indicators (e.g., discarded eligibility 

a plurality of network elements 204A, 208A, 208B, 204B, 45 bits), number of packets dropped (e.g. during a given time 

which arc part of the communication path between user 202 interval), queue length at selected circuits within the net- 

and user 214. In the embodiment of FIG. 2, each of the work element, etc. Further, any of the above described 

network elements 204A and 204B may be, for example, a parameters may be dynamically and automatically modified 

server or a router which communicate with each other via a or updated by the policy engine and fed back to desired 

wide area network (WAN) 210. The WAN may include a 5u network elements for affecting the operation or performance 

plurality of network elements (NE) 208A, 208B, which may of the network. 

include switches and/or other network elements for provid- According to the embodiment of FIG. 2, each network 

ing a communication link between element 204A and 204B. element which reports its respective operating information 

It will be appreciated, however, that the dynamic to data store 252, such as, for example network element 

feedback-based adaptive network technique of the present ss 204A, includes a local cache 276A and an event handler 

invention may be implemented in any conventional data (EH) 274A for handling event notification/registration, 

network for providing adaptive and automatic feedback Cache 276 A represents any type of memory device which 

control of network elements within that network. Thus, for ma y be used for storing and/or caching updated control 

example, the technique of the present invention may be information received at the network element. In a specific 

implemented in conventional LANs, WANs, MANs, 60 embodiment, this memory device may be a persistent store, 

internetworks, general purpose networks, packet switched After the updated control information has been cached into 

networks, circuit switched networks, etc. Moreover, the local cache 276A, the network element retrieves the updated 

technique of the present invention may be applied to any control information from the cache and re-configures itself 

conventional data network. using this updated control information. Additionally, as 

As used in this application, the term "control element" 65 shown in FIG. 2, network elements 208A and 208B may also 

refers to the additional hardware and/or software control include respective local caches (C) 276A, 27 6B for storing 

components for implementing dynamic feedback control of updated control information, and may also include respec- 
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live event handling entities (EH) for receiving/reporting network element resorts to retrieving control parameters 

event notification information. from the cache 276A- 

Although many of the various features of the present As shown in FIG. 2, network element 204A includes an 

invention are described with respect to the network element event handler 274A. In a specific embodiment, each network 

204A of FIG. 2, it will be appreciated that the descriptions 5 element which receives its control information from the data 

of the various techniques of the present invention are store includes an associated event notification agent, 

applicable to any network element in the network. However, In a specific embodiment, the event notification service is 

for purposes of simplification, network element 2 04 A is implemented using a standardized common object request 

used in this application to exemplify many of the techniques broker architecture (CORBA) event service API, commonly 

provided by the present invention. ™ known to those skilled in the art. However, although the 

One advantage of providing a local cache 27 6A for standardized CORBA event service API is generally known 
storing updated control information is that this data will be to those having ordinary skill in the relevant art, the use of 
available for use by the network element even during times a CORBA event service as a notification mechanism to 
when the communication link between the network element notify data network elements of changed network conditions 
and other parts of the network (e.g., data store 252, event 15 and/or parameters is believed to be a novel concept. In an 
server 270, policy engine 254, etc.) have been interrupted. alternate embodiment, the well known simple network man- 
Thus, for example, when the network element is unable to agement protocol (SNMP) may be used for implementing 
access information from the data store or other network event notification in the feedback-based adaptive network of 
elements, the local cache 276A may be accessed in place of the present invention. Alternatively, event notification may 
data store 252. In specific embodiments where the protocol 20 be implemented by utilizing IP sockets, 
for accessing data within data store 252 differs from the One purpose of event handler 274A is to provide the 
protocol for accessing data within data cache 27 6 A, a ability for network element 204A to register with an event 
translation mechanism may be provided. For example, server in order to receive notification for specified events, 
where the data store is implemented as an LDAP server, and such as, for example, the availability of updated control 
the local cache is implemented as a file system, a translation 25 information at data store 252. In a specific embodiment, 
mechanism will be provided at the network element for event handler 274 A may consult a configuration file 
converting LDAP calls into file system calls and vice versa. (residing at element 204A) in order to determine which 

In a specific embodiment, all current control parameters events to register with at event server 270. This feature is 

used by a particular network element (e.g. network element 3Q described in greater detail in a latter section of this appli- 

204A) are maintained within its local cache 276A The cation relating to FIG. 7 of the drawings, 

network element configures itself using the control infor- Another purpose of event handler 274A is to provide the 

mation contained within the local cache 276A. Periodically, ability for the network element to receive event notification 

when the control information for that network element is messages from event server 270. Thus, once the event 

updated (by policy engine 254, for example), the updated 35 handler has registered for event notification with event 

control information is then stored within cache 276 A. server 270, the event handler is then able to receive event 

Thereafter, the network element may reconfigure itself using notification messages from event server 270, and take appro- 

the updated control information stored within cache 276 A. priate action. This feature is described in greater detail in a 

This feature is described in greater detail in a latter section latter section of this application relating to FIGS. PAand 9B 

of this application relating to FIGS. 9B and 10 of the 40 of the drawings. 

drawings. In at least one embodiment, the cache is imple- Yet another purpose of the event handler is to monitor 

mented using non-volatile memory so that the updated specified network elements, and report the detection of 

control information may be saved in the event of a power specified events (e.g. detected errors) to event server 270. In 

failure of the network element. a specific embodiment, the event handler may consult a local 

One advantage of caching the updated control information 45 configuration file to determine the particular type of events 

within cache 276A is that the updated control information it is to be monitoring at a specified network element. In an 

may be immediately stored within cache 276A even during alternate embodiment, the event handler is statically pre- 

times when the network element itself is busy and unable to configured so that when it is initialized, it automatically 

immediately reconfigure itself using the updated control monitors a specified network element for specific types of 

information. In this way, the device which handles the data 50 events and reports detected events to event server 270. For 

transfer and caching operation is able to complete its task example, when an error is detected by network element 

without having to wait for the network element to be 204A, the event handler 274A will report the error to event 

available for re-configuration before caching the updated server 270 to be forwarded to other network and/or control 

information. elements (e.g. policy engine 254), which may be interested 

In an alternate embodiment, the current control param- 55 m wis tv P e of information. This feature is described in 

eters used by the network element are maintained in a main greater detail in a latter section of this application relating to 

memory of the network element (not shown). In this FIG - 9B of drawings. 

embodiment, cache 276A functions primarily as a back-up Although a detailed explanation of the event notification 

in case of a connection failure with other parts of the service is provided in latter sections of this application, a 

network. The updated control information are pushed 60 brief example will now be provided using the network of 

directly into the main memory of the network element and FIG. 2. 

the control action is immediately implemented. In this example, it will be assumed that network element 

Additionally, the updated control information is stored 204A uses event handler 274A to register for event notifi- 

within cache 27 6 A, which acts as a default/back-up mecha- cation service with event server 270. When the policy engine 

nism. When a communication failure occurs and the network 65 254 calculates updated control information for network 

element is unable to access the updated control information element 204A, it stores the updated control information in 

from the data store, event server, and/or policy engine, the data store 252. Event handler 272 continually monitors the 
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data store for updated control information and other events. source, caching the retrieved control information, receiving 

In an alternate embodiment (not shown) the event handler [ky ent notification messages, generating event messages 

272 may physically reside at a device other than data store delating to the status of router 10), etc. CPU 62 preferably^ 

252. When the event handler 272 detects newly updated Accomplishes all these functions under the control of soft- ^ 

control information within the data store relating to network 5/ j^are including an operating system (e.g., the Internetwork 

element 204A, it notifies event server 270 which, in turn, .*' ^Operating System (IOS®) of Cisco Systems, Inc.) and any 

notifies network element 204Aof the updated control infor- ,' jffijpropriate applications software. CPU 62 may include one 

mation. In one embodiment, the actual control information , Kr more processors 63 such as a processor from the Motorola 

is passed to the network element and cached within local mamily of microprocessors or the MIPS family of micropro- 

cache 276A. In an alternate embodiment, notification relat- jBpcssors. In an alternative embodiment, processor 63 is 

ing to the availability of updated control information is ^specially designed hardware for controlling the operations of 

passed to the network element (via event handler 274A). The 4outer 10. In a specific embodiment, a memory 61 (such as 

network element then responds to this notification by I non-volatile RAM and/or ROM) also forms part of CPU 62. 

retrieving the appropriate updated control information from jJHowever, there are many different ways in which memory / 

data store 252 and caching the retrieved data into cache '^could be coupled to the system. Memory block 61 may bey 

276A. < ^used^fora_variety of purposes such as, for example, caching . 

It will be appreciated that the above -described event ^and/or storing data, progfamminginstructionsretcr-/~^ / 

notification technique may be applied to any desired net- The interfaces 68 are typically provided as interface cards 

work element within the network. Further, it will be appre- (sometimes referred to as "line cards"). Generally, they 

ciated that any description of hardware and/or software ^ control the sending and receiving of data packets over the 

features relating to a specific network element of the present network and sometimes support other peripherals used with 

invention (such as, for example, element 204 A) may also be the router 10. Among the interfaces that may be provided are 

extended to other network elements in the network (e.g., Ethernet interfaces, frame relay interfaces, cable interfaces, 

208A, 208B, 204B). DSL interfaces, token ring interfaces, and the like. In 

Generally, the dynamic feedback-based adaptive network 2 s addition, various very high-speed interfaces may be pro- 
element of the present invention may be implemented using vided such as fast Ethernet interfaces, Gigabit Ethernet 
software and/or hardware. For example, it can be imple- interfaces, ATM interfaces, HSSI interfaces, POS interfaces, 
mented in an operating system kernel, in a separate user FDDI interfaces and the like. Generally, these interfaces 
process, in a library package bound into network may include ports appropriate for communication with the 
applications, on a specially constructed machine, or on a 30 appropriate media. In some cases, they may also include an 
network interface card. In a specific embodiment of this independent processor and, in some instances, volatile 
invention, the technique of the present invention is imple- RAM. The independent processors may control such com- 
mented in software such as an operating system or in an munications intensive tasks as packet switching, media 
application running on an operating system. control and management. By providing separate processors 

A software or software/hardware hybrid of the feedback- 35 for toe communications intensive tasks, these interfaces 

based adaptive network element of this invention is prefer- allow the master microprocessor 62 to efficiently perform 

ably implemented on a general-purpose programmable routing computations, network diagnostics, security 

machine selectively activated or reconfigured by a computer functions, etc. 

program stored in memory. Such programmable machine Although the system shown in FIG. 5 A is one specific 

may be a network device designed to handle network traffic. 40 router of the present invention, it is by no means the only 

Such network devices typically have multiple network inter- router architecture on which the present invention can be 

faces including frame relay and ISDN interfaces, for implemented. For example, an architecture having a single 

example. Specific examples of such network devices include processor that handles communications as well as routing 

routers and switches. For example, the adaptive feedback- computations, etc. is often used. Further, other types of 

based network of this invention may include specially 45 interfaces and media could also be used with the router, 

configured routers such as specially configured router mod- Regardless of network device's configuration, it may 

els 1600, 2500, 2600, 3600, 4500, 4700, 7200, 7500, and employ one or more memories or memory modules (such as, 

12000 available from Cisco Systems, Inc. of San Jose, Calif. for example, memory block 65) configured to store data 

A general architecture for some of these machines will and/or store program instructions for the general-purpose 

appear from the description given below. In an alternative 50 network operations and other specific functions described 

embodiment, the adaptive feedback-based network may be herein. The program instructions may control the operation 

implemented via a general-purpose network host machine of an operating system and/or one or more applications, for 

such as a personal computer or workstation. Further, the example. The memory or memories may also be configured 

invention may be at least partially implemented on a card to store: current and updated operating information relating 

(e.g., an interface card) for a network device or a general- 55 to the operation of the device, event notification messages, 

purpose computing deyice._— device configuration information, program instructions for 

r Fore xaro plerme ne1\^k4iement 204A of FIG. 2 may be\ implementing event notification service, program instruc- 
|a^ricr^such! asTnTrputeVlO^iUustTated in FIG. 5A of tb|/ lions for automatically and dynamically re-configuring the 

Idrawings. Referring to FIG. 5A, a router 10 is shown which device using updated control information, etc. 

|nay includes a master central processing unit (CPU) 62 j 60 Because such information and program instructions may 

interfaces 68, and a bus 15 (e.g., a PCI bus). When acting be employed to implement the systems/methods described 

under the control of appropriate software or firmware, jthe herein, the present invention relates to machine readable 

CPU 62 is responsible for such router tasks as routing table media that include program instructions, operating 



computations and network management. It may also! be information, etc. for performing various operations 



I responsible for reporting operating information (relating to 65 described herein. Examples of machine-readable media 
v^e-router 10) to one or more external devices, retrieving include, but are not limited to, magnetic media such as hard 
newor updated control information from an external data disks, floppy disks, and magnetic tape; optical media such as 
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CD-ROM disks; magneto -optical media such as floptical 
disks; and hardware devices that are specially configured to 
store and perform program instructions, such as read-only 
memory devices (ROM) and random access memory 
(RAM). The invention may also be embodied in a carrier 5 
wave travelling over an appropriate medium such as 
airwaves, optical lines, electric lines, etc. Examples of 
program instructions include both machine code, such as 
produced by a compiler, and files containing higher level 
code that may be executed by the computer using an 10 
interpreter. 
Data Store 

As depicted in FIG. 2, data store 252 provides memory for 
storing current network operating information reported from 
one or more of the network elements. The data store also 15 
provides memory for storing updated network control infor- 
mation generated by the policy engine 254. The data store 
may exist on a stand alone device which includes its own 
CPU, and operating system (e.g. Windows NT), and 
memory. Alternatively, the data store may reside as a virtual 20 
device at any one of the network elements. In a specific 
embodiment (not shown) a plurality of data store devices 
may be provided in the network. Where multiple data store 
devices are provided, the information within each data store 
may be redundant (e.g. for back-up purposes or to provide 25 
rapid access by network elements). Alternatively, specific 
data stores may service selected portions of the networks in 
which case the information within each data store may not 
be redundant. 

Each data store may be implemented as a database, a fiat 30 
file, a directory, data cache, or other data storage server 
implemented on top of some type of data storage media. In 
a specific embodiment, data store 252 of FIG. 2 is imple- 
mented as a directory server which supports the light-weight 
directory access protocol (LDAP). The LDAP protocol is 35 
commonly known to those having ordinary skill in the art. 

In a specific embodiment, the data store may be imple- 
mented using the Microsoft Active Directory (by Microsoft 
Corporation of Redmond, Wash.) or the Novell Directory 
(by Novell Corporation of Provo, Utah). One purpose of the 40 
data store in the feedback-control loop is to collect and store 
current network operating information for subsequent analy- 
sis. Collecting the operating information allows for flexibil- 
ity in the control mechanism of the feedback-based network. 

An example of data store 252 is shown in FIG. 5B of the 45 
drawings. As shown in FIG. 5B, data store 252 includes a 
CPU 252C (which may be shared with other devices or 
agents), an event handling entity 252D, and a data storage 
block 252F having memory for storing network information. 
Memory block 252A may be used for storing operational 50 
information reported by the various network elements. This 
information is used by the policy engine (254, FIG. 2) to 
calculate updated control information related to the various 
network elements. The new or updated control information 
generated by the policy engine will be stored in block 252B 55 
of the data store. The event handler 252 D monitors the data 
stored in block 252B as part of the event notification service. 
Policy Engine 

As described above, in one embodiment of the present 
invention, updated network conditions from the various 60 
network elements are constantly fed into data store 252. The 
information in the data store powers a policy engine which, 
in turn, effects changes in the network. When updated 
network conditions are fed into the data store, it is important 
that the policy engine be notified of the updated network 65 
operating information. The policy engine may either repeat- 
edly poll the data store for updated network data, or rely on 
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an event service to be notified that a change in the network 
conditions has occurred. When alerted to changes in network 
conditions, the policy engine 254 is then triggered to per- 
form its application specific tuning function. 

The policy engine 254 is a decision-making (logical) 
component of the feedback-based adaptive network of the 
present invention. An example of a policy engine 254 is 
illustrated in FIG. 5 C of the drawings. As shown in FIG. 5C, 
the policy engine 254 includes a policy server 254a, a CPU 
254g (which may reside at the policy server or other device), 
an event handling entity 254/, and a plurality of application 
specific plug-in policies 254b— e. An application specific 
policy is decision tree that allows the policy server to make 
a decision based upon measured variables and conditions. 
The definition of "policy" varies based upon the perspective 
of the user. In the case of management and control of 
network elements, a policy is a corrective action which is 
used to restore the network element to a pre-determined 
state. 

In a specific embodiment, the policy engine may be 
implemented on a stand alone device which includes CPU, 
an operating system such as, for example, Solaris 
(manufactured by Sun Microsystems of Mountain View, 
Calif.) or Windows NT (manufactured by Microsoft Corpo- 
ration of Redmond, Wash.), memory, and a storage device 
such as disk storage for storing a policy library of plug-in 
policies. Alternatively, the policy engine may be imple- 
mented as a virtual device which resides on one of the 
network elements such as, for example, a router. 
Additionally, the policy engine may be implemented with 
neural networks or with other artificial intelligence tech- 
nologies. 

The overall architecture of the policy engine is such that 
it allows for application specific plug-in policies to be added 
or deleted from the system. The policy server design is made 
modular so that decision tree modules can be added and 
deleted at will without disruption to the policy engine. Each 
application specific plug-in policy module (254fc-254e) may 
be implemented as a plug-in to the policy server 254a. 
Application specific policy plug- ins include frame relay 
policies, ATM policies, dial up policies, traffic shaping 
policies, quality of service (QoS) policies, security policies, 
admin policies, SLA policies, etc. Examples of various 
application specific policy plug-ins are shown in FIG. 5C, 
and include an ATM policy plug-in 2S4b t a frame relay 
policy plug-in 2546, a dial up policy plug-in 2546, and a 
security policy plug-in 254e. The examples shown in FIG. 
5C merely illustrate some of the various application specific 
policy plug-ins which may be attached to the policy server 
254A. 

Each application specific plug-in policy module may 
include one or more policies. For example, the frame relay 
policy plug-in 254c may include a first policy for controlling 
the CIR parameter of a frame relay circuit, and may include 
a second policy for controlling the EIR parameter of a frame 
relay circuit. Each policy contains a specific set of rules for 
analyzing specific information from selected network 
elements, and for generating updated control information in 
response to the analysis of the gathered information. The 
policy may include protocol specific rules, business logic 
rules, event notification rules, etc. Further, as described in 
greater detail below, the policies may be dynamically 
re-configured during run-time, meaning that a specific 
policy may be dynamically modified in situations, such as, 
for example, where it is determined that the current policy 
for controlling an aspect of network performance is ineffec- 
tive. 
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In a specific embodiment, the policies which form part of 
the policy engine may be stored in a policy library. Each 
policy may be implemented as a decision tree comprised of 
nodes, wherein at each node an action may be performed if 
specified criteria is satisfied. The policy may be configured 
as an application or program which runs on top of the 
operating system of the policy server. The policy engine 
traverses the decision trees of the various policies. Policies 
may be used to monitor, manage, and/or control a variety of 
network characteristics including quality of service (QoS), 
security, access, usage, etc. 

Thus, for example, the policy engine may be used to 
implement policies at one end of the frame relay link based 
upon conditions at the other end of the link. If a link that is 
downstream from a switching network experiences 
congestion, the policy engine will dynamically control the 
upstream link to adapt to the network conditions so that the 
downstream network does not drop any packets. 

FIG. 12 shows an example of a flow diagram representing 
a CIR policy (or decision tree) for a frame relay network. 
The frame relay CIR analysis policy 1200 is responsible for 
analyzing CIR information reported to the data store, and for 
generating updated CIR values based upon the analysis of 
the reported information. The updated CIR data may then be 
used by various elements in the network to modify or affect 
each element's respective behavior or operation (thereby 
affecting operation of the network). In this way, the network 
elements are able to automatically and dynamically adapt to 
changing network conditions. 

In a specific embodiment, each block within the flow 
diagram of FIG. 12 represents one or more nodes in a policy 
tree of the frame relay CIR policy 1200. The policy 1200 of 
FIG. 12 may be traversed by a policy server which has been 
configured to analyze each node in the policy tree and take 
appropriate action in response to the analysis. An example of 
a policy analysis procedure is shown in FIG. 11 of the 
drawings. 

FIG. 11 shows an example of a flow diagram of a policy 
analysis procedure U00 which may be used for traversing 
nodes in a policy tree such as the policy tree described in 
FIG. 12. At 1102, a specific application policy is loaded into 
the policy server f or execution. The application specific 
policy may be automatically loaded upon on initialization of 
the policy analysis procedure, or may be loaded subse- 
quently upon the occurrence of an event, such as, for 
example, the execution of a specific user application. In a 
specific embodiment, the policy analysis procedure 1100 is 
a multiple threaded process with each thread running its own 
application specific policy. Alternatively, the policy analysis 
procedure 1100 may be implemented as multiple processes 
or as a single threaded process with multiple concurrent 
runtime instances. It will be appreciated that either of these 
embodiments allows for multiple policy processes to be 
running concurrently in order to dynamically control various 
aspects of the network performance. 

Starting at the top of policy tree (1104), the current node 
in the policy tree is read (1106). If the current node is a read 
node (1108), then the policy server performs the specified 
read operation (1120). If the current node is a write node 
(1110), the policy server performs the specified write opera- 
tion (1122). If the current node is a script node (1112), the 
policy server runs the specified script (1124). A script node 
may be composed of multiple read nodes, write nodes, state 
nodes, computation nodes, etc. Script nodes compute the 
next state in the policy tree based upon the results of the 
computation. If the current node is a compute node (1114), 
the policy server performs the specified computation (1126). 
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If the current node is a state node (1116), the policy server 
branches to the appropriate node in the policy tree (1128). If 
the format or instructions at the current node are not 
recognized, an error is returned and the policy tree is exited 
(1118). Moreover, where an error occurs, the processing of 
the policy is aborted. 

After the policy analysis procedure has performed the 
appropriate action in accordance with the current node in the 
policy tree, a determination is made as to whether the current 
node is the last node in the policy tree (1130). If the current 
node is not the last node in the policy tree, the policy 
analysis procedure proceeds (1132) to the next node in the 
policy tree, whereupon the next node is processed as 
described above (e.g. starting at block 1106). 

If, however, the current node in the policy tree is the last 
node in the policy tree, at 1134, a determination is made as 
to whether the policy analysis procedure should be repeated 
for the currently loaded policy. In some instances, it may be 
desirable to have specific policies continuously running in 
order to provide continuous monitoring and/or updating of 
network parameters. In other instances, it may be desirable 
to have a specific policy procedure analysis concurrently 
running only while a specific user application is running. 

If it is determined that the policy analysis procedure is not 
to be repeated for the currently loaded policy, the policy 
analysis procedure is exited. However, if it is determined 
that the policy analysis procedure is to be repeated for the 
currently loaded policy, at 1136, a determination is made as 
to whether the current policy has been modified by other 
control elements in the network. As described in greater 
detail in a latter section of this application, a policy within 
policy engine 254 may be dynamically modified in order to 
alter the way in which that policy effects control of the 
network. In a specific embodiment, for example, when a 
specific policy has been modified, an associated register or 
flag may be set. At block 1136, this flag or register may be 
checked to determine if the code for the currently loaded 
policy has been modified. If it is determined that the 
currently loaded policy has been or is about to be modified, 
the modified policy is then loaded (1102) and executed by 
the policy server. If, however, it is determined that the 
currently loaded policy has not been or is not about to be 
modified, the policy analysis procedure returns to the top 
node in the policy tree (1104) and repeats the analysis 
procedure of the current policy as described above. 

By way of example, the policy analysis procedure 1100 of 
FIG. 11 will now be applied to the frame relay CIR policy 
tree of FIG. 12. The frame relay CIR policy analysis 
procedure of FIG. 12 may be used, for example, to provide 
updated CIR parameter data to frame relay network ele- 
ments 304A and 304B of FIG. 3 based upon the number of 
packets dropped on links B and C of FIG. 3. Using the 
technique of the present invention, adaptive frame relay 
parameter handling can be expanded to support complex 
situations without operator intervention. 

Referring to FIG. 12, at 1201, the frame relay CIR policy 
is loaded by the policy analysis procedure 1100. After the 
frame relay CIR policy has been loaded by the policy server, 
at 1202, various parameters are initialized. The values used 
for parameter initialization may be derived, for example, 
from a configuration file associated with the frame relay CIR 
policy. The specific parameters initialized in box 1202 
describe a threshold value (representing a maximum allow- 
able value of dropped packets for a given link), a data source 
for retrieving network element information and for writing 
updated control information (e.g., data store, network 
element), and specific formulae used for computing updated 
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CIR values based upon the number of packets dropped from 
a particular link. 

After the initialization parameters have been set, at 1204, 
the packet dropped count data for link B is retrieved from the 
specified data source and assigned to the variable X. In a 
specific embodiment, the specified data source is data store 
352. In an alternate embodiment, the specified data source 
may be network element 304A. However, for purposes of 
this example, it will be assumed that the data source is data 
store 352. Next, the dropped packet count for link C is read 
(1206) from the data store and assigned to the variable Y. 

At 1208, the dropped packet count values from links B 
and C are each compared with the threshold dropped packet 
count value (set during initialization at 1202). If the dropped 
packet count value for either of links B or C do not exceed 
the established threshold value, then it may be inferred that 
the current CIR values for links B and C are ad equate for 
achieving a desired network performance which conforms 
with predetermined criteria or specified guidelines. In the 
example of FIG. 12, network performance is determined 
using predetermined criteria which is related to the threshold 
value of the dropped packet count for links B and C. Thus, 
if the dropped packet counts for links B and C conform with 
(i.e., are less than or equal) the predetermined criteria 
specified by the threshold dropped packet count value, that 
particular aspect of network performance is determined to be 
adequate. Accordingly, the existing CIR values for links B 
and C need not be modified or updated at the present time. 
The CIR analysis then continues at block 1218 wherein the 
frame relay CIR policy procedure waits (1218) a predeter- 
mined time interval (T) (e.g. 5 sec.-30 sec.) before either 
repeating the procedure (1220) or exiting the procedure 
(1224). 

If either of the dropped packet count values from links B 
or C exceeds the threshold dropped packet count value, it 
may be inferred that at least one of the CIR values for links 
B and/or C is inadequate for maintaining a dropped packet 
count for each of the links below the threshold value. 
Accordingly, in order to adapt to this change in the network 
conditions, the CIR values for links B and C are recomputed. 

In order to re-compute the CIR values for links B and C, 
the current CIR values for these links are retrieved (1210) 
from either the data store or directly from the appropriate 
network elements. Next, the CIR value for link B is recom- 
puted (1212) based upon a specified formula which may be 
provided by, for example, a configuration file related to the 
frame relay CIR policy. Similarly, at 1214, the CIR value for 
link C is recomputed based upon the specified formula. 

An example of one formula which may be used for 
computing updated CIR values is: 

New CIR values=old CIR value*(l 

New CIR value = old CIR value* (1 + N)*K t 

Dropped_Bits/Sec 

0 < N < U and K is a predetermined 



where fraction N - 



constant (e.g. K=l) which may be dynamically adjusted to 
alter the ratio between the old CIR value and the new CIR 
value for a specified link. As explained in greater detail in a 
latter section of this application, each policy within the 
policy engine may be dynamically modified upon determin- 
ing that the current policy procedure is ineffective in con- 
trolling an aspect of the network performance to conform 
with predetermined guidelines or criteria. Thus, in the 



example of FIG. 12, if it is determined that current formula 
used for re-computing the CIR values is ineffective in 
reducing the dropped packet count for links B and C, the 
formula may be dynamically and automatically altered, for 

5 example, by changing the value of the variable K. 

Once the updated CIR parameter values for links B and C 
have been computed (at 1212 and 1214), the updated CIR 
parameter values are then written (1216) into the data store 
352. The frame relay CIR policy procedure then waits 

10 (1218) a predetermined time interval (e.g. O-30 minutes) 
before either repeating the procedure or exiting the proce- 
dure. 

After the predetermined time interval has elapsed, a 
determination is made as to whether the frame relay CIR 
15 policy procedure 1200 is to be repeated. This determination 
may be implemented using a variety of commonly known 
techniques, for example, by consulting a specific status flag, 
by polling, by receiving instructions from an external 
source, etc. If it is determined that the CIR policy procedure 
is to be repeated, at 1222, a determination is made as to 
whether the policy has been modified. In a specific 
embodiment, this determination is made by consulting a 
policy modification flag associated with the frame relay CIR 
policy. If, it is determined that the policy has not been 
modified, then the CIR policy procedure 1200 resumes at 
block 1204. 

However, if it is determined that the policy has been 
modified, the procedure resumes starting at block 1201, 
whereupon the modified policy is loaded for execution. 
Additionally, the policy modification flag may be reset at 
this time. For example, if the formula for re-computing the 
CIR values has been modified at any time during execution 
of the CIR policy 1200, a policy modification flag would be 
set to indicate that the policy has been modified. Thereafter, 
if the CIR policy is repeated or subsequently executed, the 
modified formula will be loaded at initialization and will be 
used for computing the new CIR values for the appropriate 
links. 

FIG. 4 shows an alternate embodiment of the feedback- 
based adaptive network of the present invention. The net- 
work of FIG. 4 utilizes many of the novel concepts described 
above with respect to FIGS. 2 and 3. However, one differ- 
ence in the embodiment of the FIG. 4 is that the participating 
network elements (e.g., 404A, 404B, 408A, 408B) received 
their respective updated control information directly from 
the policy engine 454. Thus, for example, in one 
embodiment, the policy engine 454 directly controls each of 
the network elements by providing updated control infor- 
mation directly to the network elements. The policy engine 
may cache the updated control information into a local data 
cache (e.g. 276A) at a network element. The network 
element may then retrieve the updated control information 
from its local cache and reconfigure itself using the retrieved 
updated control information. Alternatively, the policy engine 
may send a notification signal to selected network elements, 
advising each element that updated control information is 
available for that respective element. The network element 
may then retrieve the appropriate updated control informa- 
tion from the policy engine 454. 
60 FIGS. 6-10 illustrate examples of flow diagrams relating 
to various procedures associated with the adaptive feedback- 
based network illustrated in FIG. 2 of the drawings. 

FIG. 6 shows a specific embodiment of a flow diagram for 
an initialization procedure 600 of the present invention. The 
65 initialization procedure 600 may commence by initializing 
the various network control elements (602). As stated 
previously, the term "control element" refers to the addi- 
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tional hardware and/or software control components for 
implementing dynamic feedback control of a conventional 
network, which may include, for example, event server 270, 
policy engine 254, ADMIN system 260, monitor system 
262, data store 252, etc. Additionally, the term "network 
element" refers to any hardware or software component of 
the adaptive feedback-based network of the present 
invention, including the control elements. At 604, the 
remaining (Le. non-control) elements of the network are 
initialized. In a specific embodiment, each element may be 
initialized using existing control parameters which reside in 
a local cache (e.g., 276A) or a configuration file (not shown) 
residing at the network element. Event registration is then 
initiated (606) for the various network elements (including 
network control elements) via the event handler associated 
with each element. The event registration procedure is 
described in greater detail with respect to FIG. 7. 

FIG. 7 shows a flow diagram of an event registration 
procedure 700 in accordance with a specific embodiment of 
the present invention. Event registration for a particular 
network element may be implemented via its associated 
event handler. In the example of FIG. 7, it is assumed that 
the event registration procedure 700 is being implemented at 
network element 204A of FIG. 2. 

According to a specific embodiment, the event handler 
274A may initially consult a local configuration file (not 
shown) in order to determine which events the network 
element is to register for at the event server 270. The 
configuration file may be programmed using a command 
line interface terminal 206A. Alternatively, the event handler 
may be statically pre -configured to automatically register for 
specified events upon initialization. Thus, at 702, the event 
handler 274A determines the particular events for which 
network element 204A is to be registered at the event server 
270. Thereafter, the event handler 274A registers (704) 
network element 204A for notification of selected events, for 
example, by attaching to selected event channels at event 
server 270. As described previously, in a specific embodi- 
ment the event notification service is implemented utilizing 
a standardized CORBA event service API, commonly 
known to those skilled in the art. After the network element 
has registered with the event server for event notification, 
the network element is then ready to receive and react to 
event notification messages. Additionally the network ele- 
ment is able to report event occurrences (e.g. errors detected 
by the network element) to the event server 270, where other 
network elements have registered to be notified of such 
events. 

FIG. 8 shows a flow diagram of a network element 
reporting procedure 800 in accordance with a specific 
embodiment of the present invention. The network element 
reporting procedure may be initiated, for example, during 
initialization of the network element. As described in greater 
detail below, the network element reporting procedure 800 
is executed at the network element and provides a technique 
for causing the network element to report specified infor- 
mation of the network element to the data store. 

When the network element reporting procedure 800 is 
implemented by a specific network element, the network 
element first determines (802) the appropriate data store(s) 
for reporting its operating information. In one embodiment, 
the network element may determine the appropriate data 
stores for reporting its operating information by consulting 
a configuration file residing at the network element. In a 
specific embodiment, an automated approach using a mul- 
ticast mechanism for discovering the appropriate data stores 
may be implemented via additional software. The automated 
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approach using multicast automates the data store selection 
process and may therefore be independent of the particular 
implementation of each data store. 

Once the appropriate data store(s) has been identified, the 

5 network clement is then initialized (804) to communicate 
with the identified data store(s). In a specific embodiment 
where the data store is implemented as an LDAP directory 
service, a network element may be initialized to communi- 
cate with the data store utilizing LDAP function calls. The 
LDAP function calls are part of a standard API library which 
may be stored at the network element and used for commu- 
nicating with the LDAP directory server. In at least one 
embodiment of the present invention, the initialization of the 
network element to communicate with a data store includes 
performing a log-on procedure so that the network element 

15 is logged onto the specified data store(s). If the network 
element is unable to access (806) the identified data store, it 
will report (808) the error to event server 270, wait (809) a 
specified time period (e.g. O-30 minutes), and then re attempt 
to communicate (804) with the identified data store(s). 

20 Once a communication link has been established between 
the network element and the identified data store(s), the 
network element computes and/or retrieves (807) specific 
information for reporting to the identified data store(s). For 
example, as shown in FIG. 3 various frame relay virtual 

25 circuit parameter values (e.g., CIR, EIR, dropped packets, 
etc.) associated with network element 304A are obtained and 
reported to data store 352. 

In a specific embodiment, the type of operating informa- 
tion which the network element is to report is specified in a 

30 configuration file located at the network element. This 
configuration file may be dynamically modified using the 
feedback technique of the present invention in a similar 
manner by which the control parameters of the network 
element are dynamically modified. In an alternate 

35 embodiment, the network element is hard coded to auto- 
matically report specified information to the data store upon 
initialization. Moreover, it will be appreciated that the 
technique of the present invention may be used to dynami- 
cally modify any file, state, parameter, etc. of any network 

40 element in the network. 

After the specified operating information has been 
retrieved and/or computed by the network element, the 
network element attempts to access (810) the identified data 
store(s) for reporting the information. If any one of the data 

45 stores is inaccessible, the network element reports the error 
(808) to the event server 270 via its respective event handler, 
and wait (809) a specified time period (e.g. 0-30 minutes) 
before re-attempting to access the data store. It is to be noted 
that the network element reporting procedure 800 may be 

50 implemented as a multi-thread process whereby each thread 
is used to report network element operating information to 
a respective data store, or whereby each thread is used to 
report a specific portion of operating information to a 
specified data store. Alternatively, the network element 

55 reporting procedure 800 may be implemented as multiple 
processes or as a single threaded process with multiple 
concurrent runtime instances. If one data store is inacces- 
sible while another data store is accessible by the network 
element, each runtime instance or thread of procedure 800 

60 may take different action as appropriate. 

If the network element is able to access a specified data 
store, the network clement reports (814) its data or operating 
information to that data store, whereupon the information is 
stored at the data store. In the specific embodiment where 

65 the data store is implemented as an LDAP server, updated 
operating information provided by the network element may 
be stored in the data store by using an LDAP library call. 
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The network element then waits (816) a specified time In addition to serving as an event notification receiving 

period before re-computing and/or retrieving updated oper- device, the event handler 274A may also function as an 

ating information to be reported to the data store(s). For event notification supplying device to the event server 270. 

example, in a specific embodiment of the present invention, For example, if the communication link between the nct- 

the specified time period T (which may be referred to as a s wor k element 204A and data store 252 is down, the event 

polling interval) may range from 0-30 minutes. However, it handlcr 2 74A may send a message to the event server stating 

will be appreciated that the length of the polling interval may mat ne twork element 204A is unable to communicate with 

vary greatly and may be appropriately determined based daU st0fC 252 ^ m c ^ ^ bc to 

upon the particular network in which the technique of the yarious othef components of the system or network which 

present invention is implemented h ^ OThscribed to reccive aotification of mis typc of CVCIlt 

In at least one embodiment of the present invention, the „ . Anwm , JJL . 

network element continually report updated operating ocemtnot. For example, ADMIN system 260 may be 

information to the data store 252. In the event where the no ^ cd m °"? er * "form a network administrator ^ of the 

network element is unable to access the data store, the error probtem. Additionally, where appropriate, the ADMIN sys- 

will be reported to the event server which, in turn, will notify lem 260 ma V 1150 inform ^ 202 and ' or user 214 of the 

the appropriate control elements of the error so that the « problem. Further, the event server may also notify the policy 

control elements may take appropriate action. engine 254 and monitor device 262 of the problem. One 

Event Notification Service reason for informing policy engine 254 and monitor device 

According to a specific embodiment of the present 262 of this type of problem is to prevent these systems from 
invention, the adaptive network relies on a policy engine that using erroneous data at data store 252 to perform feedback 
uses real-time network data (e.g. state data) to make policy 20 control of the various network elements. Thus, for example, 
decisions for gene rating updated network control parameters if network element 204A is unable to report its current 
to thereby effect changes in the network. Where the network information to data store 252, the policy engine 254 should 
data to be analyzed is obtained from the data store, it is preferably be notified in order to prevent it from using 
important that the data in the data store be up-to-date at all erroneous or out-dated data (relating to network element 
times. In order for the data to be up-to-date, a notification 25 204A) in calculating new control information for the net- 
scheme is provided both from the network elements) to the work elements. The same reasoning also applies to monitor 
data store, and from the data store to the network elements). device 262. If the information within the data store is not 

As illustrated in FIG. 2, at least a portion of the network current or up to date, it is desirable that the policy engine and 

elements include an associated event handling entity or monitor device be notified in order to prevent these devices 

event handler which communicates with event server 270. 30 from using the outdated or non-current data to dynamically 

The event handler may be implement via software, perform feedback control of various network parameters, 

hardware, or a hybrid combination of software and hardware In a specific embodiment of the present invention, each 

and is configured to function as both an event notification network element which is configured to receive event noti- 

receiving entity and an event notification supplying entity fication messages uses an event notification response pro- 

for its associated network element (host device). More 35 cedure for receiving and reacting to the event notification 

specifically, the event handler enables its host device to message. The event notification response procedure may be 

receive event notification messages relating to the occur- configured as a multi-thread process which is described, for 

rence of one or more events for which the host device has example, in FIGS. 9Aand 9B of the drawings. Alternatively, 

registered to receive notification. Additionally, the event the event notification response procedure may be imple- 

handler is able to provide notification of events related to its 40 mented as multiple processes or as a single threaded process 

host device to the event server. The event server may then with multiple concurrent runtime instances, 

broadcast notice of the event to other event handling entities FIGS. 9 A and 9B illustrate a specific embodiment for 

which have subscribed or registered to receive event noti- implementing a multi-thread process of an event notification 

fication messages relating to that particular event type. response procedure. Referring first to the Event Notification 

When an event notification message is distributed to any one 45 Procedure 900A of FIG. 9 A, and using network element 

of the network or control elements, it is received by the 204A as an example, event handler 274A continually moni- 

respective event handler of that element. In a specific tors (902) selected event notification channels at the event 

embodiment, event notification service may be implemented server to which that network element has attached. When an 

utilizing an asynchronous notification scheme. event notification message is received (902) at the network 

Thus, for example, event handler 272 which is associated 50 element, the event handler then queues (904) the notification 

with data store 252 may monitor the availability of updated message in a notification queue. Although not specifically 

control information at data store 252, and notify the event shown in FIG. 2, each network element which is configured 

server 270 of the availability of such data as it becomes to receive event notification messages includes an event 

available. The event server 270 may then send event noti- notification queue or other data structure for temporarily 

fication messages to selected network elements which have 55 storing the event notification messages until that message 

registered to receive event notification relating to the par- can be processed. 

ticular updated control information which has become avail- FIG. 9B illustrates a specific embodiment of an Event 
able at data store 252. Using network element 204A as an Notification Response Procedure 900B of the present inven- 
example, the event notification message dispatched by event lion. The event notification response procedure 900B is 
server 270 will be received by event handler 274A In a 60 implemented by the network element which works in con- 
specific embodiment, the event handler queues and pro- junction with the event handling entity of that element for 
cesses each received event notification message. During responding to the event notification message. At 912, the 
processing of the event notification message, the event event notification queue is checked for any event notification 
handler notifies the network element 204A of the event messages which may have been enqueued during the pro- 
occurrence, whereupon the network element may then take 65 cedure of 900A. Assuming there is at least one event 
appropriate action. In this way, the event handler 274A notification message in the queue, the oldest event notifica- 
serves as an event notification receiving device. tion message is dequeued (913) from the queue and ana- 
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lyzed. Based on the event notification message, specific retrieved from the data store. If the retrieved values from the 

information relating to the event is determined (914). For data store are determined to be invalid, the network element 

example, where the event notification message relates to may simply ignore the updated parameter data and continue 

updated control information being available at data store to behave using its current control parameter information. 

252, at 914, the specific control parameters which have been 5 Additionally, where the values in the data store are deter- 

updatcd are determined from the event notification message. mined to be invalid, the network element may report the 

The network element then attempts to access (916) the error or problem to the event server 270, which may then 

data store 252 in order to retrieve the specified updated forward the error information to other appropriate control 

control information. In the event that the network element is elements for logging or responding to the error, 

unable to access the data store, the event notification mes- 10 Thus, at 922, a determination is made as to whether the 

sage is enqueued (917) in the message notification queue, retrieved updated control information is valid based upon 

and the error is reported (916) to the event server via the the validation preformed at block 920. If the retrieved 

network element event handler. Thereafter, flow of the parameter data is not valid, the values are discarded and an 

procedure resumes at 912, whereupon another attempt is error is reported (925) to the event server 270 for distribution 

made to fully respond to the event notification message. Io 15 to appropriate network elements. Thereafter, flow control of 

a specific embodiment, any event notification messages the procedure 900B resumes at block 912. If, however, it is 

which have been re-queued (because, for example, they determined that the retrieved parameter data is valid, at 924 

were not able to be fully processed) are processed ahead of the newly retrieved control information is written in to the 

other event notification messages in the queue. For example, local cache or memory 276A of the network element. In a 

in a specific implementation, the event notification messages 20 specific implementation, a "cached data" flag or register is 

are enqueued in order according to each message's respec- set to indicate that new or updated control information exist 

tive arrival time at the network element. within the local cache 2 76 A. Thereafter, flow control of the 

It is to be noted that the error reported to the event server procedure returns to block 912. 

270 (relating to the network element not being able to access In alternate embodiments of the present invention the 

the data store) is forwarded for broadcast to any network is event response procedure 900B may differ slightly depend- 

elements which have registered to receive notification of ing upon how the adaptive feedback-based network of the 

such events. For example, the error may be reported to the present invention is implemented. For example, in a specific 

administration system 260 in order to alert a system admin- alternate embodiment, the actual values relating to the 

istrator of the problem. Additionally, the error may be updated control information are passed in the event notifi- 

reported to the policy engine 254 in order to inform the 30 cation message to the corresponding network element. Thus, 

policy engine that network element 204A is no longer in this embodiment, the actual values relating to the updated 

reporting current information to the data store. In this way, control information may be determined directly from the 

the policy engine is prevented from generating new control event notification message upon dequeuing the message 

information based upon erroneous or outdated data located from the notification queue. Accordingly, the network ele- 

at data store 252. 35 ment need not retrieve the updated control information from 

Assuming, however, that the network element is able to the data store, 

access data store 252, at 918 the network element retrieves After new or updated control information has been cached 

the specified updated control information from data store into the local cache of a respective network element, a 

252. Any suitable mechanism may be used to retrieve the network element parameter update procedure is used to 

new values of the updated control information from the data 40 configure or reconfigure the network element using the 

store. In a specific example where the data store is imple- updated control information cached within the local cache, 

mented as an LDAP directory server, an LDAP library call FIG. 10 shows a network element parameter update 

may be used to retrieve the updated control information. procedure 1000 in accordance with a specific embodiment of 

Where the data store is implemented as a relational database, the present invention. Procedure 1000 may be implemented 

an SQL search command may be used for retrieving the 45 automatically by the network element upon determining that 

updated control information. Alternatively, where the data updated control information is available in its local cache, 

store is implemented as a file, a read command may be used Initially, at 1002, the procedure waits until it is notified that 

to retrieve the control information. Where the data store is new data has been cached into the local cache. In a specific 

accessible using a simple network management protocol embodiment this is implemented via a "cached data" register 

(SNMP), a get-parameter command may be used for retriev- 50 or flag. However, it will be appreciated by those having 

ing the updated control information. ordinary skill in the art that there are a variety of conven- 

Once the updated control information has been retrieved tional techniques for determining whether new data has been 

from the data store, the data is validated (920) as a safety cached into the local cache of the network element, 

precaution in order to ensure that the updated control If it is determined that new data has been written in to the 

parameter values are within an allowable range or boundary 55 local cache 276 A, the newly cached data is retrieved (1004). 

specific to that particular network element. In a specific The network element then automatically configures or 

embodiment, the predetermined range or boundary criteria reconfigures itself (1006) using the cached information. The 

may be stored, for example, in a configuration file residing cached information represents updated control information 

at the network element. In an alternate embodiment, the generated by the policy engine in response to changing 

updated parameter data retrieved from the data store is 60 network conditions. Typically, when the network element 

dynamically validated based upon a platform evaluation configures itself using the updated control information, its 

performed by the network element. For example, the net- performance and/or operation is affected in some manner, 

work element may execute a procedure for dynamically This effect of the performance or operation of the network 

determining upper and lower bounds for a respective control element will typically be reflected in the new operational 

parameters based upon its current operating condition and/or 65 information which the network element reports to data store 

performance. The upper and lower bounds for these param- 252. This new information will, in turn, be analyzed by the 

eters may then be used to validate the control information policy engine 254, which may generate new control infor- 
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mation in response to the newly reported information from Once the event handler has determined the specific event 
the network element. In this way, feedback control of the channels for attachment, it attaches (1504) to the specified 
network element is achieved. event channels at event server 270. In a specific 
After the network element has been reconfigured utilizing embodiment, the event handler may attach itself to the 
the updated control information, the "cached data" flag or 5 specified event channels at the event server using standard- 
other indicator used for indicating new data cached within CORBA APIs. Thereafter, at 1506, the event handler 
the local cache 276A may be reset. Thereafter, the process procedure opens a connection with the data store 252. In a 
returns to a previous state represented by block 1002 s cific cmbodimcm , the connection with the data store is 
As described previously, the adaptive feedback-based using a Java naming and directory interface 



network of the present invention includes an event notifi- 



cation service which is implemented via event server 270 1U ^^?* t . iL , 4 . . . 

and the plurality of eveni handlers associated with the * ™ th £ e da a store has been 

network and control elements. Event server 270 (FIG. 2) established the event handler is then able to retrieve specific 

may be implemented as a separate device having at least one data from data store in order to determine whether the 

CPU and memory, or, alternatively, can be implemented as specified data has been updated by the policy engine. In a 

an application to run on top of the operating system of 15 specific embodiment, the particular data retrieved by the 

another device in the network. One responsibility of the event handler is specified in the event handler programming 

event server is to manage the plurality of event channels instructions. In the example of FIG. 15, it is assumed that the 

which are used by the various event handling entities to event handler is responsible for reporting updated control 

transmit and receive event notification messages. When an information stored within data store 252. Accordingly, the 

event handler detects an occurrence of an event, the event 20 event handler will retrieve (1508) the current control param- 

handler reports the event to the event server 270 which, in eter values reported by the various network elements to the 

turn, broadcasts an event notification message to selected data store, and will also retrieve (1510) the desired or 

network elements using a specific event channel. In a updated control parameter values generated by the policy 

specific embodiment of the present invention, the event engine. For example, where the specific event relates to 

notification service is implemented using standardized 25 updated CIR values for the frame relay circuit associated 

CORBA event service APIs. with network element 204A (FIG. 2), the event handler will 

FIG. 15 shows an example of a flow diagram for a data retrieve the current CIR value reported from network ele- 

store event handler reporting procedure 1500, in accordance ment 204Ato the data store, and will also retrieve the desired 

with a specific embodiment of the present invention. Pro- CIR value for the frame relay circuit associated with element 

cedure 1500 may be implemented via the event handling 30 204A generated by the policy engine. At 1512, the current 

entity 272 residing at data store 252. One responsibility of and desired control parameter values are compared in order 

event handler 272 is to continually monitor the data store for to determine (1514) if the values have changed or have been 

new or updated control information which has been gener- modified by the policy engine. 

ated by the network elements or by the policy engine 254. If it is determined that there is no change between the 
When event handler 272 detects the occurrence or availabil- 35 current and desired values of the specific control parameter 
ity of a new control parameter, it notifies the event server being analyzed, the procedure waits (1518) a specified time 
270 which distributes the event notification message onto period (e.g., 0-30 minutes) before repeating the analysis, 
the appropriate network element(s). In a specific starting at block 1508. In an alternate embodiment, this wait 
embodiment, event handler 272 may be implemented as a cycle may be replaced by another event service routine, 
process with multiple threads. Each thread may be respon- 40 If, however, it is determined that the current control 
sible for monitoring and reporting a specific event occurring parameter value reported from the network element differs 
at data store 252. Alternatively, event handler 272 may be from the desired control parameter values generated by the 
implemented as a single threaded process which allows for policy engine, the event handler will generate (151 6) an 
multiple runtime instances to be running concurrently. For event message. In a specific embodiment, this event change 
example, one thread or runtime instance of event handler 45 message may be forwarded to the event server 270 for 
272 may continually monitor the data store for updated CIR distribution to other network elements. Thereafter, the event 
data associated with a first subset of network elements in the handler will wait a specified time period before repeating 
network. A different thread or runtime instance of event procedure 1500 starting at block 1508. 
handler 272 may continually monitor the data store for Monitor System and ADMIN System 
updated EIR parameters associated with a second subset of 50 As described previously, at least one embodiment of the 
network elements. Thus, it will be appreciated that the data adaptive feedback-based network of the present invention 
store event handler reporting procedure 1500 describes a includes a monitor system (262, FIG. 2). As described in 
monitoring and reporting technique which may be imple- greater detail below, monitor system 262 may be configured 
mented by one of the multiple threads or run-time instances to serve a variety of purposes, and therefore may be referred 
of event handler 272 . 55 to by different names, depending upon the particular rune- 
Referring to FIG. 15, upon initialization of the event tion it is performing. Like the policy engine, the monitor 
handler 272, or at some point thereafter, the event handler system 262 may be implemented as a separate machine 
determines (1502) the particular event(s) to be monitored for having memory, an operating system ( e.g. Windows NT, 
reporting to event server 270. In a specific embodiment, this UNIX, LINUX, DOS, etc.), and at least one CPU. 
information may be determined by consulting a configura- 60 Alternatively, monitor system 262 may be implemented on 
tion file associated with data store 252. Alternatively, the top of the operating system of another one of the network 
event handler may be hard coded to automatically monitor elements (including the control elements). In this latter 
specified events upon initialization. Additionally, once the embodiment, it is possible for the monitor system to share 
specific event(s) to be monitored have been identified, the the event handler device associated with that network ele- 
event handler determines the associated event channels) for 65 ment. In a specific embodiment, the monitor system 262 
attachment (to the event server) relating to the identified may be implemented in Java, and run on one of the above - 
event(s) to be monitored. described platforms. Additionally monitor system 262 may 



06/23/2004, EAST Version: 1.4.1 



US 6,584,502 Bl 

27 28 

include a plurality of business application plug-ins for within the network. In a specific embodiment, the network 

analyzing various aspects of network performance or other statistics are retrieved from the data store, 

network characteristics. The quality metrics calculation functionality is described 

In a specific embodiment, monitor system 262 utilizes the in block 1306. The network statistics or other characteristics 

policy engine 254 for running the plurality of plug-io 5 which have been retrieved from the data store are then 

applications associated with the monitor system. Thus, compared against pre-established criteria to determine 

according to one embodiment (not shown), monitor system (1308) if the current network quality (relating to the par- 

262 may be configured to run on top of the policy engine ticular aspect of the network being analyzed) is within the 

254. In this embodiment, the monitor system may not acceptable bounds as established by the predetermined cri- 

include a separate event handler device, but instead may use to teria. 

the event handler device associated with the policy engine. If it is determined that the currently analyzed network 

Each of the business application plug-ins utilized by the statistics are within the acceptable bounds as defined by the 

monitor system 262 may be used to evaluate a specific set of predetermined network criteria, the monitor procedure 1300 

characteristics of the network. For example, monitor system waits (1312) a specified time interval (e.g., 0-30 minutes) 

262 may be used to evaluate performance criteria of the 15 before repeating the analysis starting at block 1304. If, 

various network elements to determine whether the service however, it is determined that the current network statistics 

quality provided by the service provider conforms with the do not meet the specified predetermined criteria, the prob- 

standards set forth in a Service Level Agreement (SLA) lem is reported (1310) to the ADMIN system 260 for 

established between the service provider and the end user or handling. Thereafter, the monitor procedure 1300 waits 

consumer. Alternatively, using different plug-in applications, 20 (1312) a specified time interval before repeating the 

monitor system 262 may be used for performance analysis, starting at block 1304. Further examples of the 

monitoring, billing, configuration management, security network quality monitor procedures which may be run by 

violation reporting, fault management, etc. Moreover, dif- monitor system 262 are provided in the descriptions of 

ferent plug-in applications may run concurrently to allow FIGS. 16-18. 

monitor system 262 to analyze different aspects of the 25 Monitor system 262 and ADMIN system 260 work 

network simultaneously. together to provide administrative functionality to the adap- 

FIG. 13 shows a flow diagram of a network performance tive feedback-based network of the present invention, 

quality monitor procedure 1300 in accordance with a spe- Additionally, the administration system 260 provides an 

cific embodiment of the present invention. According to one external interface to remote users, (e.g., users 202, 214, FIG. 

embodiment, the network performance quality monitor pro- 30 2). Using the ADMIN system 260, a remote user is provided 

cedure is a multi -thread process which is implemented by with the ability to perform configuration and manipulation 

the monitor system 262. Alternatively, the network perfor- of the logical elements within the network. The ADMIN 

mance quality monitor procedure may be implemented as a system 260 may be implemented as a separate machine 

single threaded process which allows for multiple runtime having memory, an operating system ( e.g. Windows NT, 

instances of the process to be running concurrently. 35 UNIX, LINUX, DOS, etc. ), and at least one CPU. 

Accordingly, a variety of different network characteristics Alternatively, ADMIN system 262 may be implemented on 

may be analyzed concurrently by the monitor system 262. top of the operating system of another one of the network 

For each thread or runtime instance of the monitoring elements (including the control elements), 

procedure 1300, a specific procedure or business application FIG. 14 shows a flow diagram of an admin procedure 

plug-in for analyzing an aspect of network characteristics is 40 1400 in accordance with a specific embodiment of the 

loaded (1301) for execution. At 1302, various initialization present invention. The admin procedure 1400 is imple- 

procedures are performed, which may include locating and mented by ADMIN system 260, and may be implemented as 

attaching to a "data source" for retrieving data which is to be a multi -thread process. Alternatively, the admin procedure 

analyzed, determining the specific network parameters 1400 may be implemented as a single threaded process 

which are to be analyzed, retrieving predetermined network 45 which allows for multiple runtime instances of the process 

criteria, etc. The initialization information may be obtained, to be running concurrently. As described previously, the 

for example, by consulting a configuration file associated ADMIN system 260 may be used to dynamically modify 

with the particular business application plug-in which is specific policies within the policy engine 254. 

loaded. Alternatively, the initialization may be included In one embodiment of the present invention, policies in 

within the program instructions of the loaded plug-in appli- 50 the control of WAN services are essentially treated as static 

cation. For example, in a specific embodiment, the data entities. However, in alternate embodiments, a mechanism is 

source for retrieving the network parameter information to provided for allowing these policies to adapt to changing 

be analyzed is the data store which may be accessed, for network conditions in order to enable the network to adapt 

example, using a directory access protocol. The specific to changing network conditions. Networks that adapt to 

information to be analyzed from the data store may include, 55 changing conditions are very advantageous in fulfilling 

for example, reported QR and EI R values from each of the Service Level Agreements (SLA) in WAN networks, 

links A, B, C, D, (FIG. 2). The predetermined network Moreover, a dynamic, policy-enabled, adaptive network can 

criteria may include, for example, specific CIR and EIR be a self-healing, maintenance- free network, 

values derived from the Service Level Agreement. In a specific embodiment, there are at least three tech- 

In a specific embodiment, the network quality monitor 60 niques for achieving modification of a policy within the 

system 262 includes functionality to gather information policy engine 254. A first technique is for a remote user to 

from the data store and functionality to perform quality dynamically modify the policy via ADMIN system 260. A 

metrics calculation process. Block 1304 describes the net- second technique is for a system administrator to modify the 

work statistics gathering functionality. Basically, network policy via local terminal 266. A third technique to enable the 

statistics or other characteristics are gathered or retrieved 65 ADMIN system 260 to automatically and dynamically 

from one or more data sources. It is to be noted that the data modify a policy within the policy engine using artificial 

source may include any network element or control device intelligence and/or fuzzy logic to determine the specific 
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policy modifications. The admin procedure 1400 of FIG. 14 ence example may be abstracted to be applied to any user 

provides an example of how these various techniques are application requiring additional network resources to pro- 

implemented at the ADMIN system 260. vide a satisfactory level of quality for the application to run 

In the specific embodiment of the FIG. 14, admin proce- over a network environment 

dure 1400 is responsive to either a input from a user (1402) 5 As shown in FIG. 17, when a video conference between 

or an input from the monitor system 262 for modifying a userl and user2 is initiated (1702), the network may respond 

policy within the policy engine 254. If input is received from by initiating one or more bandwidth policies at the policy 

a user, it will be assumed (for purposes of this example) that engine 1654, and may also respond by initiating one or more 

the user has provided a request to make a specific policy policies/procedures at the monitor system 1662. Inus, at 

modification, and has included the specific policy modifi- 10 1704, the frame relay CIR policy is initiated at the policy 

cations to be performed along with the request. In this engine 1654, if this policy has not already been initiated. For 

example, a user may be user 202 of FIG. 2, or may be a example, in a specific embodiment of the present invention, 

system administrator accessing ADMIN system via local me frame relay CIR policy is loaded at initialization of the 

console 266. At 1404, admin procedure 1400 determines policy engine according to instructions contained in a con- 

whether the user has permission to modify or change the 15 figuration file. Once loaded, this policy may run continu- 

specified policy. For example, an access list may be con- ous \ y anc j concurrently with other polices and/or user appli- 

sulted to determine if the user has authority to make the cations. In an alternate embodiment, when the user 1602 

proposed policy modification. If it is determined that the initiates a video conference with user 1620, this information 

user does not have authority to make the proposed js passed to ADMIN system 1660. Recognizing that the 

modification, at 1408, an unauthorized request message is 20 video conference application may require an increase in 

returned to the user, whereupon the procedure 1400 resumes allotted bandwidth, ADMIN system 1660 instructs policy 

at block 1402. If, however, the user is authorized to make the cngmc 1554 l0 initiate the frame relay CIR policy, 

proposed policy modification the ^specified policy within the WhUc ^ frame fel aR h ^ beia iniUated b thc 

pohcy engine is modified (1406) (e.g via ADMIN system u me at 17rj4> a QR u monitor ocedure is 

260) in accordance with the proposed pohcy modification 25 concurrcntlv (1716) at monitor svstem 1662f if this 

specified by the user Thereafter, admin procedure 1400 procedure has not been mitiated . monitor sys . 

resumes at block 1402. tem J662 load ^ CIR policy monitor procedure at 

In addition to responding to input from a remote user, mitializalion m accordance with instructions contained in an 

admin procedure 1400 is also responsive to input from mitializatiorj configur ation file related to monitor system 

monitor system 262. Thus, when a reported problem from 30 m2 Mitm ^ cXyj notice of the initia tion of the video 

monitor system 262 * received (1410) at ADMIN system confereDce ap p lication may be provided by either one of the 

260, the admin procedure notifies (1412) the system admin- uscrs tQ thc aom^ system 1660 . The ADMIN system then 

istrator of the reported problem via terminal 266. The system mstnicts te monit0f tem m2 to load ^ execute the 

administrator is then able to analyze the problem and make aR mQnitor proccdurc> 

modifications to the policy engine to correct the reported 35 rt r , p , 

problem purpose of the frame relay CIR pohcy (implemented 

by policy engine 1654) is to monitor the number of packets 

ILLUSTRATIVE EXAMPLE dropped on selected links in the network, and to adjust or 

FIGS. 16-18 provide an illustrative example of how the modify the CIR parameter values on these links dynamically 

various network elements of FIG. 16 interact with each other 40 and automatically in response to changing network condi- 

to form the feedback-based adaptive network of the present tions. One purpose of the CIR policy monitor procedure 

invention. (implemented by monitor system 1662) is to evaluate the 

Referring to FIG. 16, a frame relay virtual circuit is shown effectiveness of the frame relay CIR policy. Thus, both 

connecting userl (1602) with user2 (1620). Userl commu- procedures/policies may run concurrently and, if desired, 

nicates with a first router 1604 via fink A. In the example of 45 contimi ously. 

FIG. 16, router 1604 may be managed by a first service At 1706, each of the links A, B, C, D of FIG. 16 reports 

provider (SP1). Router 1604 communicates with a frame the number of packets dropped on that link (typically 

relay cloud 1612 via link B and switch 1612. The frame reported as the number of packets dropped during a specified 

relay cloud 1610 is managed and maintained by a service time period) to data store 1652. The frame relay CIR policy 

provider such as AT&T. The frame relay cloud 1610 com- 50 at the policy engine 1654 uses this data to generate (1708) 

municates with a second router 1614 via link C and switch updated CIR parameter values for each of the respective 

1613. The second router 1614 may be managed by a second links. The updated CIR parameter values generated by the 

service provider (SP2). User 1620 communicates with router policy engine are then written (1710) into the data store 

1614 via link D. Using the network illustrated in FIG. 16, 1652. The event handler 1672 identifies the updated CIR 

various aspects of the present invention will now be 55 parameter values as new events, and notifies (1712) the 

described by way of example in which a video conference is appropriate network elements in the network of changed 

established between userl (1602) and user2 (1620). The conditions in the network. Notification of the appropriate 

video conference example is described in greater detail with network elements may be accomplished via the event server 

respect to FIGS. 17 and 18 of the drawings. 1670, which may broadcast the event notification message 

FIG. 17 shows a flow diagram of how the feedback-based 60 relating to the changed network conditions to those network 

network of FIG. 16 adapts to changing conditions in the elements which have registered to receive this type of event 

network as a video conference is initiated between userl and notification message. 

user2. A video conference application between userl and Once the appropriate network elements have been notified 

user2 is one example of a user application which may of the changed network conditions, each of the appropriate 

require additional bandwidth in order to provide a satisfac- 65 network elements may retrieve its respective updated QR 

tory level of quality for using the application to service parameter information from the data store 1652, and may 

multiple users across the network. Thus, the video confer- then update (1714) its configuration utilizing the updated 
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CIR parameter information retrieved from the data store. a specific embodiment of the present invention, the policy 

Thus, for example, the QR values used by the router one implemented by the policy engine may be allowed to 

(1604, FIG. 16) may be modified using tbc updated CIR execute a number of times before action is taken to modify 

parameter data to increase or decrease data traffic. the policy. For example, the current policy implemented by 

Presumably, the updating of the CIR parameter values in 5 the policy engine, even if judged ineffective, may be allowed 
each of the selected network elements will affect the number to execute from 0-5 hours before action is taken to modify 
ofpackets dropped on each of the links associated with these the policy. Alternatively, the current policy may be 
network elements. Thus, after the appropriate network ele- re-evaluated if it is determined that the current policy is 
ments have been updated or re-configured utilizing the improving an aspect of the network performance. For 
updated CIR parameter values, the adaptive feedback- 10 example, if the number of reported packets dropped for one 
control process resumes at block 1706, whereupon each of or more of me respective links show a reduction compared 
the linksA,B,C,D provides updated information to the data to that of previous reports, the current frame relay CIR 
store relating to the respective number ofpackets dropped at policy may be allowed to continue without modification, 
each link. This updated information is then analyzed by the Accordingly, if is determined that the current frame relay 
policy engine to generate new CIR parameter values, (if 15 CIR policy is to be re-evaluated, monitor system 1662 waits 
necessary) to be implemented by the appropriate network (1722) a specified time interval before re-evaluating the 
elements. This adaptive feedback-control process continues effectiveness of the policy. However, if it is determined that 
until the reported number of packets dropped by each of the the current frame relay CIR policy is not to be re-evaluated 
respective links conforms with predetermined criteria. When at 1726, the ADMIN system 1660 is notified of the ineffec- 
this occurs, the policy engine may still evaluate the number 20 ^ ve f rame re l av CIR policy. Thereafer, at 1728, the frame 
of packets dropped at each respective link, but it will no relay CIR policy may be dynamically modified via ADMIN 
longer need to generate new CIR parameter values since the system 1660. As described previously, the dynamic modi- 
current network parameters will reflect that the network has ficalion of a policy within the policy engine may be imple- 
adequately adapted to the current network conditions. In an mented manually by a system administrator via console 
alternate embodiment, the policy engine still generates the 25 1*>66 or, alternatively, it may be implemented automatically 
updated CIR parameter values (even if these values are the by the ADMIN system using artificial intelligence and/or 
same as the values reported by the network elements), and fuzzy logic. In the current example, the frame relay CIR 
writes these values into data store 1652. However, the event policy may be dynamically modified by modifying the 
handler 1672 recognizes that the current (i.e. reported) CIR formula which the policy uses to compute the updated CIR 
values are the same as the desired (i.e. updated) CIR values, 30 parameter values. An example of how the frame relay CIR 
and does not notify the network elements of the event. Event policy may be modified is discussed previously, with respect 
notification will be triggered when the event handler 1672 to FIG. 12. Using internal logic, the variable K (taken from 
detects a difference between the current and desired CIR the formula provided in the description of FIG. 12) may be 
values. incremented by a fixed amount (e.g., 0.1-1 .0) if it is detected 

In the CIR policy monitor procedure of FIG. 17, the 35 that the dropped packet count for a respective link exceeds 

quality monitor system 1662 (FIG. 16) may concurrently the predetermined threshold value despite the current frame 

and continuously monitor the effectiveness of the frame re lay CIR policy's attempt to reduce the dropped packet 

relay CIR policy implemented by the policy engine. In the count. 

example of FIG. 17, the effectiveness of the frame relay CIR Thus, in a specific embodiment, the policy-based adaptive 

policy is measured by analyzing the number of packets 40 control technique of the present invention continually tunes 

dropped at each of the respective links A, B, C, D, and at least a subset of network elements in the network. The 

comparing this data to predetermined criteria or guidelines. tuning of a network element may then affect the policy that 

Thus, for example, at 1718, the reported number of packets was used to tune the network device. The modified policy is 

dropped for links A, B, C, D are analyzed and compared to then used to effect future changes in the network elements, 

a predetermined threshold in order to evaluate the effective- 45 This process repeats itself as the network adapts to changing 

ness of the frame relay CIR policy implemented by the network conditions. 

policy engine. A determination is then made (1720) as to It will be appreciated that, since the policy engine 1654 

whether the frame relay CIR policy is effective in maintain- and quality monitor 1662 may each be implemented as 

ing the number of dropped packets on each or any of the multi-thread processes or single thread processes with mul- 

respective links below the predetermined threshold value. If 50 tiple runtime instances, other policies and/or quality moni- 

it is determined that the current frame relay CIR policy is toring procedures may be running concurrently with the 

effective in maintaining the number of dropped packets on frame relay CIR policy and CIR policy monitor procedure 

each of the respective links below a predetermined described in FIG. 17. An example of a concurrent quality 

threshold, the quality monitor system 1662 may wait (1722) monitoring procedure implemented by quality monitor sys- 

a specified time interval (e.g., 0-30 minutes) before 55 tem 1662 is shown in FIG. 18 of the drawings, 

re-evaluating the effectiveness of the current frame relay FIG. 18 shows an example of a flow diagram for a Service 

CIR policy by analyzing newly updated information relating Level Agreement (SLA) analysis procedure 1800 which may 

to the number of packets dropped at each of the respective be implemented by the monitor system 1662 of FIG. 16. One 

links. purpose of the SLA analysis procedure is to monitor various 

If, however, it is determined that the current frame relay 60 aspects of the frame relay virtual circuit (e.g., CIR, EIR, 

CIR policy is not effective in maintaining the number of dropped packets, Be, Be, etc.) to ensure that these param- 

packets dropped at any of the links A, B, C, D below a eters fall within specified guidelines set forth in the Service 

predetermined threshold, then at 1724, a determination is Level Agreement established between the service provider 

made as to whether the frame relay CIR policy should be and the user or client. The SLA analysis procedure may be 

re-evaluated. It may preferable to wait a specified amount of 65 implemented, for example, upon initialization of the monitor 

time in order to allow the adaptive feedback process of the system 1662. The specific guidelines or criteria set forth in 

present invention to arrive at a steady state. Accordingly, in the Service Level Agreement may be stored, for example, in 
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a configuration file residing at the monitor system 1662. A Although several preferred embodiments of this invention 

description of the service level agreement analysis proce- have been described in detail herein with reference to the 

dure 1800 is provided below using the network example of accompanying drawings, it is to be understood that the 

FIG. 16. invention is not limited to these precise embodiments, and at 

Referring to FIG. 18, at 1804, each of the links A, B, C, 5 various changes and modifications may be effected therein 

D (FIG. 16) reports its respective service level parameter *>y one skilled in the art without departing from the scope of 

data to data store 1652. In a specific embodiment of the s P irit of invention as defined in the appended claims, 

present invention, the service level parameter data is a subset What is claimed is: 

of the state data or other information reported by each of the ^ event notification system for providing event noti- 

links to the data store. The reported service level parameters 10 fication of dynamic feedback control data to network ele- 

are then analyzed (1806) by the monitor system and com- ments in a data network, the data network including a 

pared to values specified by the Service Level Agreement. plurality of network elements, each of said network ele- 

The corresponding values specified by the Service Level rnents having a plurality operating parameters associated 

Agreement may be stored, for example, in a configuration therewith, said operating parameters being related to at least 

file which is associated with the service level agreement 15 one control parameter of said element, said event notifica- 

analysis procedure 1800. lion system comprising: 

A determination is then made (1808) as to whether the at least one event supplier entity for monitoring at least 

reported service level parameters associated with the virtual onc data source for desired information relating to 

circuit being analyzed meet or exceed the corresponding updated network conditions; 

values or criteria specified by the Service Level Agreement. 20 at least one event handler for notifying an associated 

If it is determined that the reported service level parameters network element of an event relating to said updated 

do meet or exceed the corresponding values or standards network conditions; 

specified by the Service Level Agreement, the monitor an event server operable to receive information from said 

device 1662 waits (1810) a specified time interval (e.g., event supplier entity relating to said desired 

(K50 minutes) before reanalyzing the virtual circuit based 25 information, said event server being further operable to 

upon updated service level parameters reported (1804) from communicate with said event handler to cause said 

the circuit elements. event handler to alert said associated element of an 

If, however, it is determined that the reported service level event relating to said updated network conditions, 

parameters do not meet or conform with the corresponding 3Q 2. The method of claim 1 wherein said desired informa- 

values or standards specified by the SLA, a determination is tion relates to updated control information generated in 

then made (1812) as to whether the virtual circuit should be response to changing conditions in the network, 

re- analyzed using the service level agreement analysis pro- 3. The event notification system of claim 2 wherein said 

cedure 1800. There may be a variety of reasons for deciding event handler is operable to cause said associated element to 

to repeat a specific analysis procedure at monitor system 35 retrieve said updated control information. 

1662 before reporting a detected problem to the ADMIN 4. The event notification system of claim 1 wherein said 

system 1660. For example, it may be desirable to wait a event server is further operable to communicate with said 

specified time period before reporting an error or modifying event handler to register said associated network element 

a policy in order to allow the feedback-based control net- with event notification service in order to provide notifica- 

work of the present invention to adapt to the changing 4Q tion of desired events to said network element, 

network conditions and arrive at a steady state. A more 5. The event notification system of claim 1 wherein said 

detailed discussion of this topic may be found in the dis- event handler resides at said associated network element, 

cussion relating to block 1724 of FIG. 17. If it is determined 6. The event notification system of claim 1 wherein a 

that the service level parameters should be reanalyzed, the plurality of network element each include a respective event 

monitor system 1662 may wait (1810) a specified time 45 handler which communicates with said event server, 

interval before repeating the service level agreement analy- 7. The event notification system of claim 1 wherein said 

sis procedure using newly updated service level parameter at least one data source is a centralized data store, and 

data reported from the virtual circuit elements. wherein said event supplier entity resides at said data store. 

If, however, it is determined that the service level quality 8 - The cvcnt notification system of claim 1 wherein said 

of the virtual circuit is not to be re-analyzed, the ADMIN 50 evenl notification server is implemented using a common 

system 1660 is then notified (1814) of the inability of the ob i ect request broker architecture (CORBA). 

virtual circuit to conform with the criteria or guidelines 9 - ^ system of claim 1 wherein said data network is a 

specified by the Service Level Agreement. The ADMIN frame rela y network. 

system 1660 may, in turn, report (1816) the SLAproblem to 10 * A method for providing event notification of dynamic 

a system adrninistrator (via console 1666). Once alerted to 55 feedback control data to network elements in a data network, 

the problem, the system administrator may manually recon- ^ data network including a plurality of network elements, 

figure various network elements in order to rectify the each of network elements having a plurality operating 

problem. Additionally, the ADMIN system 1660 may also parameters associated therewith, said operating parameters 

report (1816) the SLAproblem to the client (e.g., user 1602, bein S relat f d to at least one . co^ol parameter of said 

user 1620) so that client may take appropriate action, if 60 element, said method comprising: 

desired. registering at least one network element with an event 

Even after the SLA problem has been reported via notification service in order to provide notification of 

ADMIN system 1660 to the system administrator and/or desired events to said network element; 

client, the monitor system 1662 may continue to perform monitoring at least one source for an event relating to 

SLA quality analysis of the selected virtual circuit by 65 changes in network characteristics; and 

repeating the service level agreement analysis procedure, as notifying the at least one network element of the event 

described in FIG. 18. relating to said changes in network characteristics. 
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11. A computer program product comprising a computer 
readable medium having computer code embodied therein 
for implementing the method of claim 10. 

12. The method of claim 10 wherein said changes in 
network characteristics relates to updated control parameters 
to be implemented by the at least one network in response 
to detection of the changes in network characteristics. 

13. The method of claim 12 further including causing the 
at least one network element to retrieve said updated control 
parameters in response to said event notification. 

14. The method of claim 10 wherein said event notifica- 
tion service is implemented using a common object request 
broker architecture (CORBA). 

15. The method of claim 10 wherein said at least one 
source is a centralized data store. 

16. The method of claim 10 wherein the data network is 
a frame relay network. 

17. The system of claim 10 wherein said changes in 
network characteristics relates to updated control parameters 
to be implemented by the at least one network in response 
to detection of the changes in network characteristics. 

18. An event notification system for providing event 
notification of network conditions to network elements in a 
data network, said event notification system comprising: 

a plurality of event handler entities, each event handler 
being associated with a respective network element; 
and 

an event notification server operable to receive informa- 
tion from each of said event handler entities relating to 



10 



20 



25 



a condition of said respective network element, said 
event server being further operable to report event 
information relating to a condition of at least one of 
said network elements to selected event handler enti- 
ties; 

each of said event handler entities being operable to 
receive event notification messages from said event 
notification server, and further operable to report infor- 
mation relating to a condition of said respective net- 
work element to said event notification server. 

19. The system of claim 18 wherein said data network is 
a frame relay network. 

20. A system for providing event notification of dynamic 
feedback control data to network elements in a data network, 
the data network including a plurality of network elements, 
each of said network elements having a plurality operating 
parameters associated therewith, said operating parameters 
being related to at least one control parameter of said 
element, said method comprising: 

means for registering at least one network element with an 
event notification service in order to provide notifica- 
tion of desired events to said network element; 

means for monitoring at least one source for an event 
relating to changes in network characteristics; and 

means for notifying the at least one network element of 
the event relating to said changes in network charac- 
teristics. 
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